[Detailed Description of the Invention] JP 2002-207992 
[0001] 

[Field of the Invention] This invention relates to an image processing method and an 
image processing device, and relates to a suitable image processing method and image 
processing device to extract an area of interest from image data with high precision 
especially. 
[0002] 

[Description of the Prior Art]As an imaging device, if it is a two-dimensional picture, 
there is an X ray imaging apparatus, for example, and if it is a three dimensional image, 
there is a CT device, an MRI device, a SPECT device, or a PET device, for example. The 
picture picturized by these devices is expressed by the concentration information about a 
certain point in a picture. For example, on a medical-application way, the existence of a 
lesion, etc. are diagnosed from such concentration information paying attention to an area 
of interest, for example, an organ. It can be called foundations of handling of a picture 
not only a medical-application way but to extract an area of interest from a picture. For 
this purpose, as shown in drawing 13, the Image Processing Division method which 
extracts an area of interest by the extraction part 1 based on the concentration information 
of the inputted image itself [ this ], and is outputted as the processing picture F from the 
inputted image A is used widely. 

[0003]As a method of extracting an area of interest from a certain picture, 
conventionally, the threshold which distinguishes an area of interest and other fields is set 
up, and the threshold process which extracts an area of interest based on this threshold, 
the edge extraction processing which extracts objective contour shape based on the light 
shade distribution shape of a picture, etc. are used widely. 

[0004]For example, in conventional technology given in JP,H7-271997,A, a function 
image is created from image data, a mask pattern is generated based on this, and the 
masked work is performed. He processes image data into dividing ridge shape a priori, 
and is trying for this dividing ridge to extract an area of interest in conventional 
technology given in JP,H11-272865,A. 
[0005] 

[Problem(s) to be Solved by the Invention]In the conventional technology given in 
JP,H7-271997,A mentioned above, there are problems - an original area of interest 
cannot be protruded, cannot extract all areas of interest, or the continuity of a border line 
is not maintained by the creation state of a mask pattern. Although this can also consider 
that conventional technology given in JP,H1 1-272865,A is a kind of threshold process 
and the continuity of the border line of the extracted field is guaranteed, In order to 
depend for the contour shape extracted on the processing method which processes image 
data into dividing ridge shape, extracting the outline of an area of interest correctly is not 
guaranteed. That is, in each conventional technology mentioned above, there is a problem 
that shape of an area of interest cannot be extracted with sufficient accuracy, the time 
series data to the part where JP,H1 1-272865,A is the same - there is also a problem that 
it is required. 

[0006]The purpose of this invention is to provide the image processing method and 
image processing device which can extract the shape of an area of interest with sufficient 
accuracy from the inputted picture. 
[0007] 



[Means for Solving the Problem]In an image processing method which extracts an area of 
interest in a picture, the above-mentioned purpose is to extract an outline picture of said 
area of interest out of said picture, to perform fitting processing with data which defined 
standard shape of said area of interest, and said outline picture, and to ask for detailed 
images of said area of interest, and is attained. 

[0008]Suitably, in the above, data which defines standard shape of an area of interest is 
described using characteristic quantity defined for every kind of area of interest, and said 
outline picture is extracted still more suitably based on a concentration value and a 
threshold of a picture. 

[0009] A 1st extraction means by which this invention extracts an outline picture of said 
area of interest out of said picture in an image processing device which extracts an area of 
interest in a picture, A database holding data which defined standard shape of said area of 
interest, A fitting means to perform fitting processing with said outline picture extracted 
by said 1st extraction means, and formed data corresponding to said area of interest read 
from said database, An image processing device provided with a 2nd extraction means to 
ask for detailed images of said area of interest based on this fitting result and the above- 
mentioned outline picture is indicated. 

[00 10] An image processing device [ fitting / this invention / with an image processing 
device / that the above-mentioned fitting means measures characteristic quantity 
extracted from said outline picture, and characteristic quantity of data of a database ]. An 
image processing device [ fitting / image processing device / that the above-mentioned 
fitting means makes either of data of a database, and said outline picture fit the other ]. 
An image processing device performs linear transform processing to one of the above, 
performs nonlinear transformation processing to one of these which made the other fit 
roughly and fitted by next, and it was made to make fit the other is indicated. 
[0011] 

[Embodiment of the Invention] Hereafter, one embodiment of this invention is described 
with reference to Drawings. Although this embodiment describes the example which 
extracts an organ from the medical-application three dimensional image obtained by a 
three-dimensional imaging device like CT or MRI, it is clear that it does not matter even 
if the object may not be a medical image, or may not be a three dimensional image and an 
area of interest is not an organ. 

[0012] Drawing 1 is a block lineblock diagram of the image processing device concerning 
a 1st embodiment of this invention. The database 4 about the shape in which an area of 
interest is standard in addition to the first extraction part 2 equivalent to the conventional 
extraction part 1 which shows drawing 1 3 this image processing device, Have the fitting 
part 3 and the second extraction part 5, the processing previous image A is inputted into 
the first extraction part 2, and the picture B is extracted, The data C from this picture B 
and the canonical form- like database 4 is inputted into the fitting part 3, the pictures A 
and B and the data D are outputted to the second extraction part 5 from the fitting part 3, 
and the after-processing picture F is outputted so that the second extraction part 5 may 
mention later for details. 

[0013]Hereafter, Image Processing Division in this embodiment is explained using the 
picture A, the picture B, the data C and the data D which are shown in drawing 5 - 
dicming 10. respectively, the picture E. and the picture F. The picture made into a 
processing object with this example gestalt is three-dimensional medical imaging, and 



extracts the blood flow which is an area of interest, and the picture of a blood vessel from 
this picture. 

[0014]The inputted picture A (dray [ny. - > receives the extracting processing of a rough 
area of interest by the first extraction part 2 first. As the extraction technique, methods, 
such as a threshold process, edge extraction processing, and a dividing ridge method, and 
those combination can be considered, for example. According to this embodiment, it 
extracts as an example by the method by the domain extension method ( drawing 2 ) which 
is the combination of a threshold process and edge extraction processing. 
rOO 151 Drawing 2 is a flow chart which shows the extracting procedure which used the 
domain extension method in the first extraction part 2. The picture A of drawmg__5 js 
expressed with the concentration value in each pixel of a picture. This picture A is a 
blood vessel picture, and is an example in which other soft tissue A 2 which lap with this 
blood vessel in part, and A 3 exist, and also diseased tissue Ai exists. A domain extension 
method performs domain extension, incorporating the connected area considered to 
belong to the same field one by one, and extracts the required whole field from starting 
point Pi set up in the area of interest. Then, starting point Pi is first set up in an area of 
interest (Step SI), Impaction efficiency is performed one after another from this starting 
point Pi, it is considered as determination object point P; for every position (Step S2), and 
that decision point P; judges whether it is the same field as the field where starting point 
Pi belongs (Step S3). 

[0016]this embodiment -- as the criteria in Step S3 -- (1) -- in the same field, the 
concentration of each pixel f (Pi) should be in a certain density range alpha. 
I A thing with small (below the threshold beta) density difference of f(P0 kalpha and (2) 
adjacent points. 

I f(P0-f(Pj) kbeta (however, Pj adjacent point of Pi) 
It judges [ whether two conditions to say are fulfilled and ]. 
[0017]The decision point which fulfills both conditions is carried out in an area of 
interest (step S4), and the decision point which does not fulfill both conditions is carried 
out the outside of an area of interest (Step S5). And when the existence of the next 
decision point is judged at Step S6, a decision point exists, it returns to Step SI and the 
decision processing of the inside and outside of an area of interest of all the decision 
points is completed, the area-of-interest extracting processing by this domain extension 
method is ended. Although the outline picture it was considered that was the same field 
may protrude a actual area of interest or may be insufficient in this domain extension 
method, rough field extraction is possible. The outline picture extracted by this first 
extraction part 2 is the picture B of drawing 6 . In the picture B, soft tissue A 2 and A 3 are 
removed, and flash field Bi which does not exist in the picture A, and insufficient field B 2 
are the examples added conversely. Diseased tissue Ai remains as field Bi as it is. 
[0018]In addition to the extraction outline picture B, the fitting part 3 receives the 
canonical form- like data (it is henceforth called the data C (drawing 7 ).) about an area of 
interest from the canonical form-like database 4. 

[00 19] When making the organ of a human body into an area of interest temporarily, it is 
assumed that this area of interest differs in shape or a size, for example according to a 
race, sex, age, etc. In other pictures and areas of interest, it is considered easily the same 
way that shape and a size have dispersion. Therefore, fitting between the picture B 
extracted as shape of the outline of an area of interest, and the data C which is canonical 



forms-like. 

[0020]For example like the picture A and the picture B, the data C which defines the 
shape of a canonical form may be data of composition so that it may have a concentration 
value in each pixel, and here, Or it may be the engineering CAD data or data like a 
mathematical theory which defines the shape of a subject, and they may be the both 
further. 

[0021]This embodiment describes the case where it is given as an example by formed 
data or a mathematical theoiy for which the data C is used by engineering CAD. By this 
method, the standard shape of an area of interest is described using the set of required 
characteristic quantity as the data C. 

r00221 Drawing 3 (i) and (ii) are the explanatory views about the characteristic quantity of 
a blood vessel and a blood flow. Simply, since the blood vessel as shown in drawing 3 
(ii) is carrying out tubed shape, in order to describe a blood vessel, distribution etc. of the 
thickness of the blood vessel wall in alignment with the solid angle from distribution, a 
strain, and the center line and center line of a radius of the blood flow which met a center 
line and this center line, for example should just be defined. It is also considered that the 
center line of a blood vessel is expressed with a certain function form in which the curve 
to draw contains an unknown with the elasticity of a blood vessel wall, etc. These 
unknowns, radius distribution, thickness distributions, etc. are characteristic quantity. The 
function f of the radius r (x, y) and the function g of thickness d (x, y) express the 
characteristic quantity Z in drawing 3 (D. 

[0023]In the fitting part 3, fitting is performed by extracting and comparing the parameter 
(characteristic quantity) which describes formed data, i.e., shape. How to fit the data C in 
the picture B on the occasion of fitting, Two kinds of the method of fitting the picture B 
in the data C can be considered, and although whichever may be sufficient, it is necessary 
to memorize the procedure of returning the picture B to the original shape since the area 
of interest which should be extracted will be transformed when it fits the data C. 
r00241 Drawing 3 explains signs that the picture B was fitted in the data C. The canonical 
form-like data C received from the canonical form-like database 4 is a set of the 
characteristic quantity which describes the standard shape of the target area of interest. 
The fitting part 3 fits by extracting the set of the same characteristic quantity as the data 
C from the extracted image B. For example, the curve which a center line draws can cut 
each part of a blood vessel into round slices, and can obtain it by connecting the central 
point obtained from there, and said unknown can be determined by fitting the function 
defined by the data C. It can obtain from the section of a round slice similarly about the 
radius of a blood vessel, and thickness. By extracting the set of such characteristic 
quantity from the picture B can describe the blood vessel within the picture B, and the 
shape of a blood flow (this is henceforth called the data D). An example of the data D is 
shown in drawing 8 , and it is set to characteristic quantity Z={f (x, y) and g (x, y). f (x, y) 
is a function of the radius r, and g (x, y) is a function of thickness here. 
[0025]The second extraction part 5 shown in drawing 1 r eceives the inputted image A 
(drawing 5 ) and the extraction outline picture B ( drawing 6 ), and the data D ( drawing 8 ) 
from the fitting part 3. Although the shape of the area of interest of the inputted image A 
has already been obtained as the data D by the fitting part 3, since the data D is defined as 
strictly standard shape, the lesion of organs, such as a tumor, is not extracted, for 
example. Then, final extracting processing is performed by comparing the picture B with 



the data D by the extraction part 5. 

[0026]The comparison with the picture B and the data D can consider taking difference, 
for example, the picture acquired by (difference is henceforth called the picture E by 
taking the difference between two pictures.), as shown in drawing 9 , Partial (it is 
henceforth written as heights) B 2 protruded in the first extraction part 2 and insufficient 
partial (it is henceforth written as crevice) B 3 are extracted as E 2and E 3 , and also lesion part 
Bi is extracted as Ei, and these partial Ei, E 2 , and E 3 are emphasized. The judgment of 
whether Ei which is this heights and crevice, E 2 , and E 3 carry out lesions, such as that it 
is a problem of the accuracy of the first extraction part 2 or a tumor, extraction 2 is 
performed based on diagnostic imaging software or human judgment. 
[0027]For example, if based on human judgment, since the position of heights or a 
crevice is identified by the picture E, the radiologist should just judge whether the part is 
a lesion or that is not right by the picture A. It removes about part E 2 which is not so, and 
E 3 , adds to the data D about part Ei judged to be a lesion, and is considered as the 
outputted image F ( drawing 10 ) with data as it is about the other part. 
[0028]On the other hand, not only the above methods but diagnostic imaging can also be 
performed by software using the data D which is a set of characteristic quantity, as 
opposed to the picture which is small rapidly by place Gi which has a radius of the blood 
vessel in alignment with a medial axis as shown for example, in drawing 4 (ii) — drawing 
4_(i) — since characteristic quantity (radius r) is dented greatly, it obtains a diagnostic 
result distinguishes this automatically and it is considered that is strangulation of a lesion, 
for example, a blood vessel, i so that it may be shown. Or if it is a blood vessel when the 
data D has a remarkably different tendency as compared with the data C, an automated 
diagnosis, like it is too thick and too thick can also be carried out, for example. Of course, 
there may also be an example which displays the above-mentioned waveform and human 
being diagnoses. 

[0029] As mentioned above, according to this 1st embodiment, since the area of interest in 
an inputted image can be extracted with sufficient accuracy, diagnostic imaging can be 
performed using an extracted image. 

r00301 Drawmg 1 1 is a lineblock diagram of the image processing device concerning a 
2nd embodiment of this invention. Although the composition of this embodiment is the 
same as a 1st embodiment fundamentally shown in drawing 1 , it serves as data in which 
the data stored in the canonical form-like database 8 has a concentration value over each 
of the same pixel as the picture B unlike a 1st embodiment. For this reason, the 
processing method in the fitting part 9 of this embodiment differs from a 1st embodiment. 
Hereafter, only processing of the fitting part in this embodiment is explained. 
[0031]Linearity or a nonlinear method, and the method that combined them as an 
example in this embodiment although the combination was known are explained to fitting 
of data with a concentration value, for example. 

[0032]How to fit the data C in the picture B like a 1st embodiment on the occasion of 
fitting, Two kinds of the method of fitting the picture B in the data C can be considered, 
and although whichever may be sufficient, it is necessary to memorize the procedure of 
returning the picture B to the original shape since the area of interest which should be 
extracted will be transformed when it fits the data C. This embodiment explains how to 
fit the picture B in the data C. 

r00331 Drawing 12 is a figure explaining the fitting method of of the data C and the 



picture B which define the standard target shape. As opposed to the data which obtained 
the data performs linear transform to the data C and the picture B is made fitting [ data / 
this embodiment / first ] roughly (Step S20), next was obtained at this step S20, It carries 
out by what the data G performs nonlinear transformation and the picture B is made 
fitting [ data ] in detail is obtained for (Step S21). 

[0034]Here, linear transform is a process which changes the data C with a total of nine 
parameters of three advancing side by side, three rotations, and three elasticity as shown 
in drawing 14 , and is made to fit the picture B roughly. 

[0035]With nonlinear transformation, two image data for fitting is divided into a small 
region, respectively, The node of the data C is moved according to the picture B, 
measuring the similarity of each node defined by division between two pictures, It is the 
method of making it transform the whole data C into the shape near the picture B 
(Reference documents: MEDICAL IMAGING TECHNOLOGY Vol.16 No.3 MAY 1998 
pp.175-183). 

[0036]An example of nonlinear transformation is explained using drawing 15 . The data C 
which fitted the picture B roughly by linear transform is divided into the small region of 
the shape of a lattice as shown in drawing 15 (i). If the node lois temporarily moved to 
the node 1 1 as shown in drawing J : 5_(ii), the field 12, for example, the field of the slash 
part in a figure, containing this node will change, as shown in the field 13. The point 
inside the field 13 corresponding to each point inside the field 12 can be searched for by 
linear interpolation here. The similarity of the pixel value in that field can be found by 
comparing the image comparison B with the field 13 after this modification. Here, 
similarity with the picture B in the field 13 is set to L (13). If each node is moved like the 
node 1 1, it can ask for the similarity L (n) about all the fields that change as a result. 

of a node is repeatedly performed so that total sigma[ of said similarity L (n) ] L 
(n) may be made into the maximum. At this time, the moving range of a node is restricted in 
order to hold the connectivity of each field. For example, in drawing 15 (ii), it is 
restricted to the range shown by the position dotted line 14 of the node 11. 
[0037]According to the above techniques, the data C and detailed fitting with the picture 
B are possible. In this technique, if the points of comparison do not exist on the picture B 
near a certain node on the data C, that node does not move, but the shape of a canonical 
form which the data C originally has passes through it, and it is saved. Therefore, fitting 
which is not influenced by the portion or the insufficient portion which were protruded 
among the areas of interest roughly extracted in the first extraction part 2, or the lesion is 
possible.The data C which changed in the fitting part 9 is henceforth called the data G. 
The second extraction part 5 receives the picture A and the picture B, and the data G, and 
obtains the outputted image H by the same processing as the first working example. 
[0038]As mentioned above, according to this embodiment, it becomes possible to be able 
to extract the area of interest in an inputted image with sufficient accuracy, and to 
perform high-precision diagnostic imaging using an extracted image. 
[0039]The image processing method of this invention extracts a customer's area of 
interest based on the data requested by the customer, is possible also for applying to a 
service enterprise which models and delivers this extracted picture using a solid shaping 
apparatus, and is not limited to medical diagnoses. 
[0040] 

[Effect of the Invention] According to this invention, it becomes possible by using the 



standard shape of the target area of interest to extract the shape of an area of interest with 
sufficient accuracy from the inputted picture. 



[Translation done.] 



